Appendix E. ITRC Groundwater Statistics and Monitoring Compliance Team Survey Results

E.1 Survey Summary

The web-based Groundwater Statistics and Monitoring Compliance (GSMC) Survey was conducted using Survey Monkey during the summer of 2011.

• ITRC received 126 responses from 34 states.
• Most questions could be skipped. The only required questions were the state and role category questions.
• The highest participation was from the states as seen in Table E-1.

Table E-1. States represented in the survey responses

States Represented
State	Responses
CA - California	15
IN - Indiana	12
FL - Florida	11
NY - New York	9
WA - Washington	8
AZ - Arizona	7
VA - Virginia	6

 

• Table E-2 includes the responses by role category.

Table E-2. Survey responses by role category

All Responses
Category	Responses
State regulator	55
Federal employee, non-regulator	33
Consultant	21
Federal regulator	7
State employee, non-regulator	5
Industry representative	3
Public/tribal stakeholder	2

 

• Table E-3 includes the respondent answers for their technical backgroundNatural or baseline groundwater quality at a site that can be characterized by upgradient, historical, or sometimes cross-gradient water quality (Unified Guidance). (individuals could choose more than one response).

Table E-3. Skill set of respondents

Skill Set
Response	Responses
Hydrogeology	60
Geology	57
Environmental science	51
Project management	38
Environmental regulations and laws	36
Remediation technology	28
Chemistry	27
Engineering	26
Groundwater modeling	19
Risk assessment	19
Statistics	18
Biology	12
Toxicology	8
Geostatistics	6
Soil science	5
Physics, Health Physics	1
Community relations	1

 

E.2 Agency- or Program-Level Responses

The survey included questions that referred to the agency or program with which the respondent either worked (if a state or federal employee) or the agency or program with which the respondent mainly interacted. Results for several of these questions are presented.

• Respondents were asked if their program or agency has specific policies or guidance for the use of statistics for groundwater data. Figure E-1 summarizes the responses.

Figure E-1. Responses to the question: Does your program or agency have specific policies or guidance for the use of statistics for ground water data?

 

• Respondents were asked about who primarily reviews or performs statistical analyses within their program or agency. This question had “please check all that apply” and "other" write-in space. The four answers that were given are listed in Table E-4. The write-in answers were categorized approximately by one of the four responses, plus the “groundwater statistics are not used” answer. A few representative comments are included below the table.

Table E-4. Responses to question about who performs or reviews the statistical analyses

Who Performs or Reviews Statistics?
Response	Number
Technical support group - in house	56
Each project manager	40
Technical support available through outside contractor	26
One specific expert who reviews or performs statistical analyses	21
Groundwater statistics not used	4

 

• Who performs or reviews statistical analyses?

  • Generally, each project manager reviews the project and consults, as needed, with the "specific expert."
  • Statistics are assigned to staff geologists, regardless of their background with statistics.
  • The technical support group is made up of chemists, engineers, and geologist. There is not a statistician in house.
  • We do not use statistical analysis in the remedial programs.

 

• Respondents were asked: If groundwater statistics are not generally used, are any of the following factors? Table E-5 includes the responses to this question.

Table E-5. Factors as to why statistics are not used

Statistics Not Used
Response	Number
Staff is not proficient in rigorous statistical analyses	35
Regulations do not require	24
No perceived need	21
Insufficient time and resources to review submittals	21
High cost of doing statistical evaluations	8
Regulations do not allow	5

 

• Respondents were asked about how challenging it is to use statistics for decision making. In addition, write-in answers describing the challenges were requested. Figure E-2 summarizes the responses for how challenging is it to use statistics.

Figure E-2. Responses for how challenging is it to use statistics.

• The identified challenges included the following:

  • checking the accuracy of the data, applicability of the method to the data set
  • knowledge of reviewer or project manager and keeping current on statistical applications if they don’t use every day; practical training; knowing when the presentation or results of statistical analyses are inaccurate or inappropriate
  • understanding of results by nonstatisticians
  • data that are suitable for statistical analyses (for example, poor planning, sparse data, not following DQOdata quality objective process)
  • misapplication of statistical analyses
  • knowledge of statistical software applications, formatting and exporting data for use in software
  • expense of software
  • expense of groundwater monitoring programs needed to generate the needed data set
  • not challenging, statistics not used in some or all programs

   

• Respondents were asked about the types of sites where policies or guidance are applicable. Figure E-3 summarizes the responses for this question.

 

Figure E-3. Types of sites for which policies or guidance are applicable.

E.3 Individual-Level Responses

The survey included a group of questions about the individual's knowledge and experience with statistical analyses.

• Figure E-4 summarizes the responses about individual knowledge and experience.

 

Figure E-4. Individual knowledge and experience with statistical analysis.

 

• Respondents were asked: How familiar are you with the USEPA’s 2009 Unified Guidance? Figure E-5 includes the responses to this question.

 

Figure E-5. Responses on familiarity with the USEPA's Unified Guidance.

 

• Respondents were asked: How do you use groundwater statistics? Figure E-6 summarizes the use of groundwater statistics by various tasks and project life cycle stages.

Figure E-6. Use of groundwater statistics by various tasks and project life cycle stages.

• Respondents identified the types of statistical tests that they use in reviewing groundwater data. A list of tests/methods identified by survey respondents is included in Table E-6 in order of most responses to fewest responses. In addition, several respondents indicated that they do not use or review any statistics or provided non-method-specific responses.

Table E-6. Statistical methods that are used or reviewed by survey respondents

Statistical Methods	Number
Simple descriptive statistics (such as sample meanThe arithmetic average of a sample set that estimates the middle of a statistical distribution (Unified Guidance)., medianThe 50th percentile of an ordered set of samples (Unified Guidance)., rangeThe difference between the largest value and smallest value in a dataset (NIST/SEMATECH 2012).)	67
Tests for simple trends (such as Mann-Kendall)	60
Confidence Intervals	50
Histograms	42
Box plots	37
Prediction limits	35
Tolerance limits	34
Normal probability or quantile plots	29
Two-sample hypothesis tests (such as t-tests)	29
Control charts	29
Multi-sample hypothesis tests (such as ANOVA)	24
One-sample hypothesis tests	18
Nonparametric Wilcoxon rank sum test	1
Outlier tests (such as Dixon's or Rosner's)	1
Principal component analysis - correspondence analysis (PCA-CA)	1
Shewart-CUMSUM test	1
Time Series (ARIMAautoregressive integrated moving average) discriminant analysis nonlinear modeling of data	1

 

• Respondents were asked about the challenges or misuses of statistics. The answers included statements such as the following:

  • lack of knowledge
  • connection between conceptual site model (CSM)A living collection of information about a site which considers factors such as environmental and land use plans, site-specific chemical and geologic conditions, and the regulatory environment (ITRC 2007b)., monitoring network, data quality, and data set collected
  • terminology issues and misunderstandings
  • drawing conclusions from insufficient data
  • ensuring that statistical assumptions are valid
  • using an overly simplified approach
  • for some software applications, a steep learning curve, expensive training, and expensive software
  • interpreting the uncertainties associated with statistical methods and tests
  • misrepresentation of the results, biased presentations
  • how to handle nondetect values
  • application of the wrong statistical test
  • issues with background
  • difficulties for the regulators in trying to recheck the calculations performed or using software to confirm that calculations were performed correctly
  • using statistics to avoid cleanup
  • software applications that are deceptively easy to use, but difficult for novice users to ensure that the calculations are appropriate
  • communicating the results to decision makers

• The respondents were asked to assign value to the topics that the GSMC team identified for the guidance document. Figure E-7 includes the responses.

Figure E-7. Value of topics for the guidance document.

 

• The respondents were asked about the value of training for their agency or organization. These responses are included in Figure E-8.

 

Figure E-8. Applicable level for training on identified topics for groundwater statistics.

Publication Date: December 2013